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Abstract: Consider a random vector {X, Y) and let m{x) = E{Y\X = x). 

We are interested in testing Hq : m € -M&^g = {7(',S>9) : 6 £ 0,9 £ (?} 

for some known function 7, some compact sot C IRP and some function 

fvj , set Q of real valued functions. Specific examples of this general hypothc- 

^ i sis include testing for a parametric regression model, a generalized linear 

QQ ' model, a partial linear model, a single index model, but also the selection of 

explanatory variables can be considered as a special case of this hypothesis. 
— sj ^ To test this null hypothesis, we make use of the so-called marked em- 



pirical process introduced by [1] and studied by [16] for the particular case 
of parametric regression, in combination with the modern technique of em- 
pirical likelihood theory in order to obtain a powerful testing procedure. 
The asymptotic validity of the proposed test is established, and its finite 

^~« ' sample performance is compared with other existing tests by means of a 

f^^ , simulation study. 
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1. Introduction 

Assume that the data {Xi,Yi) [i ~ 1, . . . ,n) are independent replications of a 
random vector {X,Y), where X is a d-variate vector and Y is one-dimensional. 
Let m{x) = E{Y\X ~ x) ior x <E M . We are interested in testing 

Ho -.me Me^g = {ji-,9,g) : G 6,3 G 5} (1.1) 

for some known function 7, some compact set Q C M^ and some function set Q of 
real valued functions. Special cases of this general null hypothesis include testing 
for a parametric model (in which case to(-) = 7(-, 9)), a generalized linear model 
{m{x) = j{f3*x,a) with 9 = (a,/?)), a partial linear model {m{x) = z*0 + g{w) 
with x — {w, z)), but the test procedure can also be used for e.g. the selection of 
explanatory variables. Other possibilities not included in the paper are testing 
for the parametric form of the variance function, the comparison of regression 
curves, etc. 

The idea of the test procedure we develop in this paper is to make use of 
the so-called marked empirical process introduced by [4] and studied by [16], 
combined with the modern technique of empirical likelihood theory in order to 
obtain a powerful testing procedure. 

A lot of research on goodness-of-fit tests based on empirical process ideas has 
been carried out in the last ten years. Starting with the already-mentioned pa- 
per by [4] and [IG], devoted to testing parametric regression models, the theory 
was continued with the problem of checking generahzed hnear models (see [20]), 
and the selection of variables (see [3]), etc. See also [25] for testing, using the 
empirical process ideas, other models of interest: partial linear models, reduction 
of the dimension, models with multidimensional response and heteroscedasticity 
tests. For the calibration of the critical points associated to the test statistics, 
two approximations were mainly used: one is based on the bootstrap (see [17]) 
and the other is based on martingale transformations (see [19]). For alternative 
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testing procedures based on smoothing techniques, see [23], and for recent gen- 
erahzations of the empirical regression process ideas to dependent data, see the 
papers by [IS], [G], [7] and [8]. 

The empirical likelihood (see for example the book by [14]) is a technique 
designed to construct a nonparametric likelihood for parameters of interest in 
a nonparametric or a semiparametric setting with nice properties typical for 
the parametric likelihood, as for example the Wilks' theorem and the Bartlett 
correction recently proved by [1]. 

There has also been some recent interest in goodness-of-fit tests based on the 
empirical likelihood in the regression context. [9] propose a sieve empirical likeli- 
hood test for testing general varying-coefficient regression models. [1 1] study the 
properties of the empirical likelihood in the presence of both finite and infinite 
dimensional nuisance parameters as well as when the data dimension is high. 
[2], [21], [12] and [5] propose different tests based on empirical likehhood with 
conditional moment restrictions, including situations with dependent data. 

In this paper we present a new procedure for testing different regression 
models in a unified way, combining ideas of empirical regression processes and 
empirical likelihood. See the paper by [10] for a different unified vision of the 
theory for testing regression models. 

The paper is organized as follows. In the next section, the unified testing 
procedure is explained, and the asymptotic limit of the test is given, under a 
set of primitive conditions. In Section 3 these primitive conditions are verified 
for a number of particular regression models. Section 4 contains the results of a 
small simulation study. The proofs are deferred to the Appendix. 

2. General test procedure 

To explain the idea of the proposed testing procedure, we first focus on the 
simple case where we test for the parametric form of the regression function, i.e. 
m{-) = 7(-,0), and where X is assumed to be one-dimensional. Note that the 
true value ^o of 6 satisfies in that case 

E[I{X<x)(Y-jiX,eo))]=0 (2.1) 

for all X G M. Let 9 be an estimator of 6o under Hq, obtained from e.g. least 
squares minimization. The idea of [4] and [IG] is to define the marked empirical 
process based on residuals 



-1/2 



J2l{X,<x)[Y,-jiX,J)] (2.2) 



and to test Hq by constructing Kolmogorov-Smirnov, Cramer-von Mises, smooth 
and directional tests based on this process. [Ki] also obtained the limiting dis- 
tribution of the proposed tests. In this paper we will extend the above process 
to a general framework including a large number of common regression models. 
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and we will make use of empirical likelihood theory in the construction of the 
test statistic. 

Let's consider this now in more detail. For the general null hypothesis Hq 
defined in (1.1), an appropriate extension of the above marked empirical process 
is given by 

i?„(M)=n-i/2^i(X„^i,^,g)[y,-7(X,,^,g)], (2.3) 

for u belonging to a compact set U, for certain estimators 6 and g depending 
on the model under Hq, and for a certain appropriate weight function L also 
depending on the particular form of Hq . In Section 3 we will consider the process 
Rniu) in detail for a number of particular models (parametric, generalized linear, 
partial linear, etc.), which will give rise to particular forms for the function L 
and the set U . In many of our examples, U will be equal to Rx, the support of 
X, which we also assume to be compact. 

We will now introduce the empirical likelihood test based on the marked em- 
pirical process i?„(w). For any bivariatc distribution F defined on the support 
of {X, F), define the likelihood L{F) by (we restrict for simplicity to the defini- 
tion for d = \, the extension to general d being straightfoward, but notationally 
more heavy) 

71 

L{F) = \{[F{X,, Y,) - F{X,-, Y,) - F{X„ Y,-) + F{X,-,Y,-)]. 

4=1 

Next, define the empirical likelihood ratio by (for any rf > 1) 

EL{u,e,g) 

^ sup{L(F) : EF[L{X,u,e,g){Y ~ 7(^,^,3))] = 0} 
sup{i(^)} 

^'ip { nr=i «'-«'»>o.Er=i '^i=i'Er=i w,L{xuu,e,g)[yr-i{XrfiM=o\ 

(2.4) 

where the maximum over the empty set is defined to be zero, and let ^(w, 6, g) 
= ~2\ogEL{u,9, g). The proposed test statistics are now given by 

Sn = snp£{u,e,g), T^ = j e{u,6,g) dfiniu), (2.5) 

for some measure ^„ such that 



sup \fJ-n{u) - n{u)\ = Op(l), 
ueu 



/, Van Keilegom et al. /Empirical likelihood based testing for regression 585 

where the measure n does not depend on n. When U — Rx, a natural choice 
for /i„ is 

dfXni-)=w{-)dFx{-), 

where w{-) is an appropriate weight function and Fx{-) is the empirical distri- 
bution function of X. The measure /i(-) equals dfi(-) = w{-)dFx{-) in that case, 
withFx(-) =P(X< •)■ 

In order to obtain the limiting behavior of the process i{u^6,g) (u € U), 
use will be made of the results in [11]. In that paper the authors give primitive 
conditions under which the empirical likelihood statistic converges (for a fixed 
u). Their result can be easily generalized to the current situation of a process 
in u. We will show this below. For this purpose, let i^{U) be the space of all 
bounded functions from U to IR equiped with the supremum metric. 

Let ^0 be the true value of 9 € Q, and go be the true function in Q. The 
result gives the asymptotic behavior of the test statistics Sn and T„ both under 
Hq and under the fixed alternative hypothesis Ha ■ m ^ A^e.e- 

Theorem 2.1. (a) Assume that under Hq, 

(CO) P{EL{u,9,g) = for some uEU)^Oasn^oo. 

(CI) The process Rn{u) can he written as 

Rn{u) = n""^/^I]r=i QiXi:Y,,u,9o,go) + op{l), uniformly in u e U , 
for some function Q defined on Rx xlRxUxQxQ that satisfies 
E[Q(X,Y,u,9o, go)] ~ for all u, and Rn{u) converges weakly to a 
zero-mean Caussian process V{u) (u G U ). 

(C2) sup„gy |n-iEr=i L^{X.,,u,e,g){Y, - ^[X,,9,g)Y - T{u)\ -^p for 
some function T{u) (u S U ), such that miueu T(u) > and 
s^PueuT{u) < oo. 

(C3) sup^^u miiXi<i<n\L{X^,u, 9, g){Y, - ^{Xi,e, g))\ =op{n^/^). 

Then, under Hq, the process l(u, 9, g) (u g U ) converges weakly in 11°° [U) 
to W'^{u), where W{u) = V{u)/y^T{u), and hence 



Sn ->d sup W'iu), Tn ^d / W'{u) dn{u). 

(b) Assume that under Ha, condition (C2) holds, and in addition 

(CI') n^'^l inf„g;7Q \Rn{u)\] = Op(l) , for a subset Uq of U of positive 

fi-measure. 
(C3') sup„gymaxi<,<„|i(X„u,^,g)2(r, -7(X„^,5))2| =op(ni/2). 

Then, under Ha, for any c > 0, 

P(5„ > c) ^ 1, P(T„ > c) ^ 1, 
as n tends to infinity. 
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Note that condition (CI') is basically saying that for some values of u, Rn{u) 
is biased. Indeed, if Rn{u) is unbiased for all u, then \R„{u)\ ~ Op{l) for all 
u € U and condition (CI') does not hold in that case. Also note that condition 
(C3') is stronger than (C3). Consider the common case where L{X,u,0,g) is a 
bounded function, and write 

Then, the second term can be taken care of using standard arguments, and 
hence it follows from Lemma 11.2 in [14] that (C3') holds true if the fourth 
moment oiY — ^{X,6,g) is finite, whereas (C3) only requires the finiteness of 
the second moment. 

Since the limiting distribution of Sn and r„ are rather complicated, we pro- 
pose to work with the following bootstrap approximation. From Theorem 2.1(a) 
it follows that the log-likelihood function £{u, 9, g) can be approximated by 
Rn{u)^ /T(u) and that Rn{u) can be written as 

n 

R„{u) = n-i/2 J2 Q{X„ Y„u, 9o, go) + op(1), 

uniformly in u G U. Define random variables Vi, . . . ,Vn that are independent 
of the data {Xi,Yi) {i = l,...,n), and that are independent and identically 
distributed such that E{Vi) ~ 0, Var(Vi) = 1 and \Vi\ < c < oo for some finite 
c. Then, let 

n 

i?:(M) = 71-1/2 ^Q(x„y„M, 00, ffo)^"». 

i=l 

From the multiplier central limit theorem (see e.g. [22], p. 179), it follows that 
Rn{u) converges to the same process as the original process i?„(u). Since the 
process i?* (u) contains unknown quantities {9o , go and possibly also the function 
Q itself), we cannot use it in our bootstrap procedure, and we therefore define 



i?:(7.)=n-i/2^Q(X„r„«,^,g)F„ 



where Q is a suitable estimator of the function Q, whose definition depends on 
the particular model at hand. We will give the precise definition of this estimator 
for each of the models we will consider in Section 3. Now, suppose the following 
assumption holds true : 

(C4) sup^gy iRUu) ~ RUu)\ = op(l). 

Then, i?* (w) also converges to the same process as the original process _R„ (u) . 
This way of approximating the process i?„(u) by means of the bootstrap has 
been used in a number of other papers, see e.g. [3], [25], among others. 

It remains now to define an estimator of the variance T{u). Let 



f(M) = n-i^L(X„u,^,.g)(r,-7(^.,^,.g))' 
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Since condition (C2) assures that sup„ |r(u) — T{u)\ = op(l), we have the 
following result concerning the bootstrap approximation of the process W'^{u), 
ueU : 

Theorem 2.2. Assume (C0)-(C4). Then, under Hq and conditionally on the 
data {Xi,Yi) (1 < i < n), the process R^(u)'^ /T{u) converges weakly in £°°{U) 
toW^iu). 

As a consequence, the critical values of the test statistics 5'„ and T„ can be 
approximated by the (1 — Q:)-th quantilcs of the distributions of 

^: = sup^|i^ 

ueu T{u) 
and 

Tn = / ^ dHn(U), 

Ju 1 (u) 
respectively, given the data (Xj, Yi) {\ < i < n) . 

3. Application of general test to specific models 

In this section we consider a number of particular models and apply the test 
procedure developed in Section 2 to each of these models. 

3.1. Parametric models 

Consider the null hypothesis Hq : m(-) = ^{-,0) for some ^ Q, where X is 
supposed to be one-dimensional (d = 1). As we have seen in Section 2, [4] and 
[IG] introduced the marked empirical process defined in (2.2). We focus here on 
the following slight variation of (2.2) : 



<(x) =n-i/2^/(x, e j,)[y, -7(x„0)], 



1=1 



where x S Rx, and 



, {i:i>a;} x<a , . 

^" " 1 {t:t<x} x>a, ^ ' 



for some a in the interior of Rx (e.g. a is the median of X). Note that Rn{x) 
equals Rn{x) with L{X, x, 6, g) ~ I{X e J^) and with 7(2;, 6, g) being a function 
of 9 only. 

The marked empirical process proposed by [Ki] accumulates data from left to 
right (i.e. it is based on the indicator I{X < x)), whereas we replaced I{X < x) 
by I{X G Jx), in order to obtain more stable results for small values of x. Note 
that when we take a = infja; : 2; £ Rx}, the left endpoint of the support of X , 
we obtain the process studied by [IG]. 
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To show the vahdity of conditions (C0)-(C3) wc will use the results in [16]. 

Note that [2] also use an empirical likelihood approach to test the appropri- 
ateness of a parametric regression model. In contrast to our method, they use a 
local approach, based on comparing a kernel estimator of the regression function 
m{x) with an estimator of m(x) under the null model. As a consequence their 
results depend on the choice of a smoothing parameter. 

Let r(x) = E{Va,r{Y\X)I{X e J^)]. The following conditions are needed for 
the main result. Note that the choice a = inf{:r : x G Rx} (i-c the process of 
[IG]) is excluded by assumption (PI). 

(PI) E{Y^) < cx), M^^R^ T{x) > and sup^g^^ T{x) < oo. 
(P2) Under Hq, 9 admits an expansion 

n 
9-00= n-^ Y. HX^,y^,So) + Op(71-l/2), 

2=1 

where E[h{X,Y,9o)] = and H{9o) := E[h{X,Y,9o)h*iX,Y,9o)] exists and is 
positive definite. 

(P3)(i) For each x e Rx, 'y{x,9) is continuously diffcrcntiable at each 9 in 
int(e), and sup^g^^ |7(a:;, 6*0)1 < oo. 

(ii) The components of the vector 4^7(x, 9) are bounded, uniformly in x G Rx 
and 6* G e. 

Condition (P2) is satisfied for the least squares estimator and any of its 
common robust modifications. 
Also, define 

G{x,9)^ J ^j{u,9)dFx{u). 

Theorem 3.1. Assume (P1)-(P3). Then, under H^ , conditions (C0)-(C3) 
hold true for 

Qix,y,x,9) ^ I{x e J,)[y ~ jix,9)] ~ G\x,9)hix,y,9) 

and V{x) a zero-mean Gaussian process with covariance function 

Cov{V ixi),Vix2)) - E[Var{Y\X)IiX G J,, (1 J^,)] + G'{x,,9o)H{9o)Gix2,9o) 
~G\xuOo)E[I{X G J,,)iY-j{X,9o))h{X,Y,9o)] 
-G\x2,9o)E[I{X G J^,){Y --i{X,9Q))h(X,Y,9o)]. 

In order to apply the bootstrap procedure described in Section 2, let R^* {x) = 
n~^^^ Er=i Q(^«' Y,,x, 9o)V, and RP*{x) = n-i/2 ^«^^ Q{X,,Y,, x, 9)V„ where 
the random variables Vi are as in Section 2, 

Qix,y,x,9) ^ I{x € J,)[y - ^ix,9)] ~ G\x,9)h{x,y,9), 
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G{x, 6) = n^^ X^iLi H^i G Jx)-§g^{Xi, 9), and h is an appropriate estimator of 
the function h satisfying 

(P4) n"i/2 J2^^^[h{X,, r„ 0) - h(X,,Y,,0oM - op{l). 

We also need to assume that 

(P5) For each x £ Rx, jix, 9) is twice continuously differentiable with respect 
to the components of 9 at each 9 in int(O), and all the partial derivatives of 
order two are bounded, uniformly in x G Rx and 9 E Q. 

Theorem 3.2. Assume (P1)-(P5). Then, under Hq , condition (C4) holds 
true, and hence the process R^*{x), x £ Rx has the same limiting distribution 
as the process R^{x). 

Note that condition (P4) is satisfied when e.g. 7(2;, 9) = 6q + 9ix, and 6 is the 
least squares estimator of 9, since in that case (we consider only 9i and write 
h{x,y,9) = {ho{x,y,9),hi{x,y,9)Y) 

hi{x,y,9) = -^{x - fix){y - IJ-y) —{x-^ix)'^, 

where fix = E{X), fiy = E{Y), a\ = Var(X) and axY = Cov(X,r). Define 
h{x, y, 9) by replacing /ix, l^v ■, (^x ^^'^ <^xy by their sample versions. Then, the 
left hand side of condition (P4) can be decomposed in a sum of i.i.d. terms, plus 
a remainder term. Consider e.g. the following term (the other terms are similar 
and lead to the same asymptotic order) : 

X X i^i 

since E[{X, - pLx){Y, - fiY)V,] = E[{X, - fix){Y, - fiY)]E{V,) ^ 0, and since 
ai-ai = Op(n-V2). 

Other bootstrap procedures can be used as well (see e.g. [17] for a wild 
bootstrap procedure). 



3.2. Generalized linear models 

Under this model it is assumed that Hq^'^^ : m(-) = 7(/3*-,a) for some 9 = 
(a,/3) G 6 C 1?°+'', d being the dimension of X. [2U] proposed the follow- 
ing marked empirical process (except for the indicator 10* Xi G J„), which is 
replaced by I{$*'Xi < u) in their paper) 

n 

for some estimators d and /?, which is of the general form given by Rn{u) with 
L{X,u,9,g) corresponding to I{(3*'X G J„) and U = {f3QX : x G Rx}- The 



/, Van Keilegom et al. /Empirical likelihood based testing for regression 590 

set Ju is as in (3.1), with a belonging now to the interior ol the set U. The 
motivation for this process comes from the fact that under H^'"'^^ , E{I{PqX G 
Ju)[Y - 7(/3*X,ao)]} = for aU u. 

Let T{u) = E[Yai{Y\f3lX)I{l3lX e J„)]. The following assumptions are 
needed for showing the validity of (C0)-(C3). 

(GLMl) E{Y'^) < oo, miueuT{u) > and snp^^i,T{u) < oo. 
{GLM2) Under Hq, {a, (3) admits an expansion 

n 

1=1 

where E[k(X, Y, ao, /?o)] = and K{ao, Po) ■= E[k{X, Y, a^, Po)k\X, Y, ao, A)] 
exists and is positive definite. 

{GLM3){i) For each x e Rx, 7(/3*a;,a) is continuously differentiable at each 
(a,/3) in int(e), and sup^^^.^ \j{/3lx,ao)\ < cxj. 

(ii) The components of the vector „, „. 7(/3*g;, a) are bounded, uniformly in 
X € Rx and (a,/?) G 6. 

(GLM4) The function E[Yai{Y\X)I{j3*„X G J„)] is uniformly continuous in 

ueu. 

Also, define 

G(u,a,/?) =E\—^^il3*X,a)Iil3'X G J„) 

Theorem 3.3. Assume {GLMl)-{GLM4). Then, under H§^'^^ , conditions 
(C0)-(C3) hold true for V{u) a zero-mean Gaussian process with covariance 
function {Oq = (ao,/5o)) 

Cov{Viui),Viu2)) = E[VariY\(3lX)IiPlX &Ju,nJu,)] + G'{ui,eo)Ki9o)G{u2,eo) 
-G'{ui,9o)E[I{PlX G J„J{y-7(/3*X,ao)}fc(X,y,0o)] 
-G'{u2,0q)E[I{(31X G J„J{y-7(/3*X,ao)}fc(X,y,0o)]- 

For showing the validity of the bootstrap approximation, first note that 
R^^^iu) = 71-1/2 ^n^^ Q(^Xi, Yi, u, Oo) + op(l), uniformly in ueU, where 

Q{x,y,u,9) =/(/3*a; G Ju)[y - j{/3*x,a)] ~ G\u,a, I3)k{x,y,a, 13). 
Define 

Q{x,y,u,e) =I{l3*x G Ju)[y-l{f3*x,a)] - G\u,a,P)k{x,y,a, (3), 
where G{u, a, (3) is the sample version of G(u, a, (3) and k[x, y, a, (3) satisfies 
{GLMh) n-^/^Y.l=l[Hx^,Y,,aJ) - k{X,,Y,,ao,l3o)]V, = op{l). 
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Also, assume that 

{GLM6) For each x G Rx, ^{P^x.a) is twice continuously diffcrcntiable with 
respect to the components of (a,/?) at each (a,/3) in int(0), and all the partial 
derivatives of order two are bounded, uniformly in a; G Rx and (a, /3) G 0. 

Theorem 3.4. Assume {GLMl)-{GLMQ). Then, under H^^"' , condition (C4) 
holds true, and hence the process R^^^^*{u) = n~^" Y!^7=i Qi-^ij ^i? ■": S)Vi, has 
the same limiting distribution as the process i?^^*^(it), u ^ U. 

3. 3. Selection of variables 

For the purpose of this subsection we write X = [Xi, . . . , Xd) = [W, Z), where 
W, respectively Z, is a random vector of dimension d^, respectively dz, and 
d = dyj + dz- Also, write Rx = Rw x Rz- Consider the null hypothesis Hq^ : 
E{Y\X) = g{W), where g{w) = E{Y\W = w), i.e. the response depends only 
on the covariate vector W and not on Z. [3] considered this testing problem 
and used the fact that under H^^ , E{I{X S Jx)Iw{W)[Y - g{W)]} = for 
all X G i?x, where fw is the probability density of VF, to propose the following 
test statistic : 

n 

R^Jix) = n-i/2^/(X, G J.)fw{W,m-g{W,)], 

where J^ = O^i J^j^ each J^j is as in (3.1) for some Oj in the interior of the 
support of Xj, I{X, G Jx) = I{X.a E Jx^,-- ■,Xid & JxJ, 



UH^inh'-rY.^{^^)^ (3-2) 



n „, 

gH = /v^H-i(n/i''-)-i^i^(l!l^)r„ (3.3) 



1=1 

-^(^) ~ rii^i ^'(%)j fc is a univariate kernel and h a bandwidth. Note that 
as before, we have replaced the indicator I{Xi < x) by /(X; G Jx), but the 
fundamental idea of [3] remains the same. 

Define T{x) = E[Yar{Y\X)f^{W)I{X G Jx)] and q{z\w) = P{Z G Jz\W = 
w), and consider the following assumptions : 

(SVl) fw e T^, g e Tl and q{z\-) e T^ for all z e Rz and for some 
A,T, w > 0. 

{SV2) k : IR -^ M is an even function of uniformly bounded variation, sat- 
isfying k{u) ~ 0((1 + |m|"+^+^)~-'^) for some e > 0, Ju'^k{u)du = 6io, for 
i = 0, . . . , a— 1, where Sij is Kronecker's delta and a = £+t—l, with i—l<X<i 
and t - 1 < T < t. 
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{SV3) {nh''"')-^ + n/i2min(r,A+i) _, Q as n -> oo. 
(SF4) E{\Y ~ giW)\^+^} < cx) for some 6 > 0. 
{SV5) inix^R^ T{x) > and sup^^j^^, T{x) < c». 

Assumptions {SV1)-{SVA) are taken from [3], and ensure the validity of 
condition (CI). The class J- a (for a, /? > 0) defines a class of smooth functions, 
satisfying certain moment conditions. We refer to [3], p. 1474, for its formal 
definition. 

Theorem 3.5. Assume {SV1)-{SV5). Then, under H^^ , conditions (CO)- 
(C3) hold true for V{x) a zero-mean Gaussian process with covariance function 
(where Xj = {wj, Zj), j = 1, 2j 

CoviVixi),Vix2)) = E^iY ^ g(W)ff^(W)I{W e J^,n J^,) 

X[I{Z e J,J - q{z,\W)][I{Z e J,J - q{z2\W)]y 

For the bootstrap approximation, [3] showed that R^^{x) ean be written as 

n 

R'J{x) ^ n-i/2 Y, {I{X^ e J.)fw{W,){Y, - g{W,)) - 

i=\ 

- r{x, Wi)fw{Wi){Y, - g{Wi))\ + op(l), 
where r{x,Wi) = I{Wi £ Ji,^q{z\Wi), and they proposed to estimate r{x,Wi) 

by 

f(x,iy.) = 1 yA-( ^\^0 /(X, E J.). 

nh^^f^{W.OJ^^ y h J ' ' '' 

The consistency of the bootstrap approximation obtained by multiplying each 
term in the above representation by Vi , and by replacing the unknown functions 
in this representation by their corresponding estimators, has been established 
by the same authors. We refer to their paper for details about the proof and the 
assumptions under which this approximation is valid. 

3.4- Partial linear models 

Wc continue to write X = {Xi, . . . , Xd) = {W, Z) as in the previous subsection, 
and consider now the following null hypothesis : 

HP'^:E{Y\X) = 9lZ + g{W), 

i.e. Y = 9qZ + g{W) + e, where E{e\W, Z) = 0, and where the function g is 
completely unknown. This problem has been studied by [3] and [2-''>]. Note that 
under H^^ , E{Y -m{W)-e^o[Z -mz{W)]\X} = 0, where m{W) = E{Y\W) = 
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9*oE{Z\W)+g{W) and mz{W) = E{Z\W}. This suggests to define the foUowmg 
marked empirical process : 

n 

<^(a;) = n-i/2 ^ j^x, e J:,)w{W,){y, - m{W,) - 9'[Z, ~ ifiziW^)]}, 

i=l 

X E Rx, where w{-) is an appropriate weight function (e.g. w{W) = fwiW)), 

i=\ 

-^(^) ~ rifci^l'^j)) ^ is a univariate kernel, h a bandwidth, and jw{) and 
m(-) are defined as in (3.2) and (3.3) respectively (except that we now use the 
notation m(-) instead of g(-))- Finally, the estimator B is defined by 

n 

e = S-^n~^ J2 ^'(^0[^» - mz{W,)][Y, - m{W,)], 

i=\ 

where 

n 

S = n-^Y1 w\W,)[Z, - mz{W,)][Z, - rhz{W,)]K 

i=l 

Define T{x) = E[Va.r{Y\X)w'^{W)I{X e J.^)], and consider the following as- 
sumptions : 

(PLl) Conditions (l)-(6) given in [2~>], p. 72-73, are valid. 

{PL2) infj;gflj^, T{x) > and sup^^g^j^ T{x) < oo. 

The conditions in [25] guarantee the weak convergence of the process R^^{x) 
{x G Rx)- They consist of certain smoothness conditions on m{w), mz{w) and 
q{z\'w) = P{Z G Jz\W = w), conditions on the bandwidth h and on the kernel 
k, moment conditions on Y, Z and e, and conditions on the weight function w. 

Theorem 3.6. Assume {PL1)~{PL2). Then, under H^^ , conditions (CO)- 
(C3) hold true for V{x) a zero-mean Gaussian process with covariance function 

Cov{V{xi),V{x2)) = E{e^w^{W)D{X,xi)D{X,X2)}, 

where 

D{X, x) = I{X G J,) - q{z\W)I{W G J„,) 

-e[w{W)I{X G J.,)[Z - mz{W)f^S-^[Z - mziW)], 

and S = E{w'^{W)[Z - mz{W)][Z - mz{W)Y}. 

The proof of this result is very similar to the one of Theorem 3.5 and is 
therefore omitted. 

The consistency of the bootstrap procedure described in Section 2 is estab- 
lished in Theorem 5.3.1 in [25]. We refer to this book for the assumptions under 
which this approximation is valid. 
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3.5. Other models 

The procedure developed in this paper can also be applied to a number of other 
situations : testing for homoscedasticity ([2()]), the comparison of regression 
curves ([13]), single index models ([24]), autoregressive models ([18]), testing 
for conditional independence ([3]), among others. In each of these situations, a 
suitable marked empirical process has been proposed and its asymptotic behav- 
ior has been developed. In a similar way as we have done for the applications 
worked out in Subsections 3.1-3.4, one can use these convergence results to show 
the validity of the proposed empirical likelihood tests. We leave the details to 
the reader. 

4. Simulations 

We present some simulations carried out to assess the behavior of the new test 
for moderate sample sizes. We consider situations under the null, which allows to 
assess the accuracy of the bootstrap approximation, and under the alternative, 
to show the power. The new test is compared with the original test based on the 
marked empirical process. The simulated examples were chosen for the problem 
of testing a parametric regression model, and for a generalized linear model with 
binomial response. 

4.1. Parametric model 

The data are taken from the following regression model 

Y, = X, + d (X,) + a {Xi) e, i G {1, . . . , n}, 

where n is the sample size, Xi is a uniform random variable on the interval [0, 1], 
Ei is a standard normal random variable, the parametric model to be tested is 

Hq : m{x) ~ Ox 

for some parameter 9 to be estimated, d is a function which represents the 
deviation from the null hypothesis and a is the conditional standard deviation 
of li given Xi. The simulations were carried out under the null hypothesis, d = 0, 
and for different alternative functions d. Homoscedastic models were considered 
by means of a constant function cr, and several heteroscedastic models were also 
studied by means of different functions a. 

The following tables contain the percentage of rejections for ten thousand 
samples, under the nominal level 5%. The results are given for different values of 
the sample size, n, for different alternative functions d and for different standard 
deviations of the error. The functions d are coded as 0. d{x) = 0, 1. d{x) = 
x^, 2. d{x) == 0.3a;exp(x), 3. d{x) ^ 0.3 sin(47ra;) and 4. d{x) = 0.4xl{x < 
0.5) — 0.4(1 — x)I{x > 0.5). The standard deviation is coded as 1. cr(x) = 
0.25, 2. a{x) = 0.5x and 3. a{x) = 0.125(2 - x). The columns "IRF-KS" and 
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"IRF-CVM" contain the percentage of rejections for the test by [l(>], with the 
Kolmogorov-Smirnov and Cramer-von Mises statistics, respectively. In order to 
approximate the critical values for this test, wild bootstrap was used, as it is 
explained in [17]. The columns "EL-KS" and "EL-CVM" contain the percentage 
of rejections for the new test, with the Kolmogorov-Smirnov and Cramer-von 
Mises statistics, respectively. The sets were constructed using the value a = 
0.5. In order to approximate the critical values, the bootstrap proposed in this 
paper was used. Five thousand bootstrap replicates were used for bootstrap 
approximations . 

Table 1 
Percentage of rejections in homoscedastic models 

a d n IRF-KS IRF-CVM EL-KS EL-CVM 








50 


5.35 


5.09 


7.07 


5.90 







100 


5.09 


5.11 


5.80 


5.27 




1 


50 


73.07 


68.51 


73.07 


76.30 




1 


100 


95.90 


93.69 


95.23 


96.72 




2 


50 


31.76 


28.26 


32.65 


33.94 




2 


100 


57.69 


51.71 


55.07 


59.50 




3 


50 


46.37 


37.65 


37.49 


35.78 




3 


100 


79.60 


68.76 


70.06 


66.85 




4 


50 


40.72 


29.87 


50.09 


32.06 




4 


100 


73.73 


54.25 


81.24 


56.79 



Under the null, the percentages of rejections arc close to the nominal level 5% 
in all cases, the main deviation coming from the empirical likelihood tests with 
sample size n — 50, while for n ~ 100 the percentages are already close to 5%. 
Under the alternative, the power of the empirical likelihood tests is generally 
higher than the power of the original tests. This is the case for models 1, 2 and 
4, while for model 3, a sinusoidal alternative, the power of the orignal test is 
higher. The improvement obtained by means of the empirical likelihood idea is 
larger under heteroscedastic models. 

Table 2 
Percentage of rejections in heteroscedastic models 

cr d n IRF-KS IRF-CVM EL-KS EL-CVM 



2 





50 


6.11 


5.82 


9.01 


7.02 


2 





100 


5.33 


5.17 


6.94 


5.43 


2 


1 


50 


76.00 


83.85 


85.67 


83.77 


2 


1 


100 


96.08 


98.52 


98.40 


98.14 


2 


2 


50 


34.57 


39.60 


45.69 


42.02 


2 


2 


100 


58.89 


66.76 


67.29 


65.82 


2 


3 


50 


48.13 


47.67 


37.01 


37.50 


2 


3 


100 


79.27 


79.76 


68.06 


65.10 


2 


4 


50 


37.48 


40.79 


68.25 


41.22 


2 


4 


100 


72.26 


70.67 


95.69 


67.92 
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Table 3 
Percentage of rejections in heteroscedastic models. 

a d n IRF-KS IRF-CVM EL-KS EL-CVM 



3 





50 


4.92 


4.93 


6.84 


5.68 


3 





100 


4.70 


4.51 


5.20 


4.86 


3 


1 


50 


84.88 


79.23 


84.54 


87.91 


3 


1 


100 


99.38 


98.19 


99.21 


99.64 


3 


2 


50 


41.28 


36.15 


42.22 


46.12 


3 


2 


100 


71.62 


63.10 


70.04 


75.65 


3 


3 


50 


53.50 


42.86 


46.42 


43.23 


3 


3 


100 


88.54 


78.65 


88.65 


83.21 


3 


4 


50 


56.09 


38.69 


64.88 


43.68 


3 


4 


100 


87.60 


66.19 


92.66 


72.83 



4-2. Generalized linear model 

The simulated responses Yi are taken from the binomial distribution with 15 
trials and the probability of success p{Xi) for i G {1, . . . ,n}, where n is the 
sample size and Xi is a uniform random variable on the cube [—1,1] x [—1,1] x 
[0,2]. 

The generalized linear model to be tested is 

Ho : p{x) = 7 (/?*x) 

where 7(0:;) = e^/(l + e^) is the logistic transformation and (3 is some parameter 
to be estimated. 

For the simulated model under the null, the parameter (3 was defined to be 
(3 = (1,2,0.5)*. Two alternatives were considered: p{x) = $(/3*x) where $ is 
the standard normal distribution function, and (3 = (1,2,0.5)* as before, and 
p{x) = 7(2:1 + 2x2 + 0.25(.T2 + 1)^) with 7 the logistic transformation. We call 
them a probit model and a quadratic logistic model, respectively. 

Table 4 contains the percentage of rejections for ten thousand samples, under 
the nominal level 5%. The results are given for different values of the sample 
size n, and for the three models. The five columns named "M-IRF-CVM" con- 
tain the percentages of rejections for the test by [20], which is a test based on a 
martingale-transformed Cramer-von Mises statistic. The parameters were esti- 
mated using the maximum likelihood method. For the transformation, smooth- 
ing is necessary in order to estimate the function E{X\f3*X = u). To this aim, 
a Nadaraya- Watson estimator was used with Epanechnikov kernel on the in- 
terval [—1,1], and five possible values of the bandwidth: 1.0, 1.5, 2.0, 3.0 and 
4.0. The five columns in Table 4 contain the results for these five bandwidths, 
respectively. 

The columns "EL-KS" and "EL-CVM" contain the percentage of rejections 
for the test proposed in this paper, with the Kolmogorov-Smirnov and Cramer- 
von Mises statistics, respectively. The sets were constructed using the value 
a = 0.5. In order to approximate the critical values, the bootstrap based on 
multipliers, as proposed in Section 2. was used. Five thousand bootstrap repli- 
cates were used for bootstrap approximations. 
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Table 4 












Percentage of rejections 


in models with binomial 


response 






n 




M-IRF-CVM 




EL-KS 




Model 


1.0 


1.5 


2.0 


3.0 


4.0 


EL-CVM 


Null 


50 


4.87 


4.89 


5.04 


4.97 


5.04 


7.53 


5.70 


Null 


100 


4.19 


4.08 


4.15 


4.13 


4.03 


7.01 


5.31 


Null 


500 


4.84 


4.84 


4.75 


4.66 


4.63 


5.47 


4.87 


Probit 


50 


0.33 


0.31 


0.31 


0.27 


0.31 


13.32 


11.86 


Probit 


100 


0.40 


0.42 


0.38 


0.38 


0.37 


13.33 


12.96 


Probit 


500 


7.51 


7.50 


7.44 


8.02 


8.73 


21.10 


36.85 


Quadratic 


50 


8.36 


8.06 


8.23 


7.93 


8.03 


12.79 


12.34 


Quadratic 


100 


13.44 


13.18 


12.71 


12.25 


12.20 


16.44 


19.84 


Quadratic 


500 


68.05 


66.60 


64.44 


62.31 


62.34 


49.53 


75.69 



These results show a good approximation to the level under the null. The 
biggest deviations come from the Kolmogorov-Smirnov statistic of the empiri- 
cal likelihood process with small sample size, although the result for n = 500 
shows the consistency of the bootstrap. The power is much larger for the em- 
pirical likelihood test, especially with the Cramer-von Mises statistic. Note that 
the martingale-transformed statistic is based on a Cramer-von Mises statistic. 
The difficulty to detect the probit alternative should also be emphasized, with 
percentages of rejections even smaller than the nominal level for the martingale- 
transformed statistic. By other way, the bandwidth has not much influence on 
the results for the martingale-transformed statistic. 

Appendix: Proofs 

In this Appendix we prove the main result (Theorem 2.1) that states the generic 
conditions (C0)-(C3) under which the convergence of the test statistics is guar- 
anteed, and we also verify these conditions for the models considered in Sec- 
tion 3. 

Proof of Theorem 2.1. 

(a) The result is an easy extension of Theorem 2.1 in [11], with a„ = 1 and 
h = {9,g). The extension lies in the fact that their result is limited to the 
convergence of £(w, 6, g) for a fixed value of u, whereas we consider a process 
in u. It is however a straightforward exercise to show that under conditions 
(C0)-(C3), the process l{u,e,g) {u G U) converges to W{u) in i°"{U). The 
convergence of the test statistics 5'„ and r„ now follows immediately. 

(b) We will show that P(S'„ > c) ^ 1 as n tends to infinity. The proof for 
T„ is similar. Write P{Sn > c) = P{c < Sr, < oo) + P{Sn = cx)). We will 
show that P{c < 5'„ < oo) — P(5„ < cxa) -^ 0, from which it then follows that 
P{Sn > c) ^ 1. If Sn < c)o, then for each u ^ U, the supremum in (2.4) is a 
supremum over a non-empty set, and hence it is easy to show that 



£(u, ^, g) = 2 ^ log (l + A„(u)A,(u) 



i=l 
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where the Lagrange multiplier A„(u) satisfies 



Et 



+ Xn{u)Ai{u) 



-0, 



(A.l) 



and Ai{u) = L{Xi,u,9,g)[Yi — ^{Xi,9,g)] {i ~ l,...,n). First, we will show 
that under Ha, and provided S'n < oo, 



-V2( inf \X„iu)\) =op(l), 

\uEUo / 



i.e. for all ?7 > 0, 



Pin^l'^ inf |A„(m)| >?/ 



5'n < Oo) ^ 1, 



(A.2) 



where Uq is the subset of U defined in condition (CI'). Note that it follows from 
(A.l) that 



= ^A,(^.)-A„(^)^- 



A\(u) 



i=l 



and hence 



n^l'^ inf |A„(m)| = n^/^ inf 

uGC/o ueUo 



^1 + A„(u)yl,(w)' 



EtiMn) 



> 



"-^Er=i[A'H][i + A„(^)A,(^)]-i 

n-i/2sup„.JA,(«)P ' 



since n ^ J^^^ii^ '^ ^ri{u)Ai{u)] ^ = 1, and this goes to infinity, using (CI') and 
(C3'). Hence, (A.2) holds true. Next write 



Mu) , ,2,..^^ A^u) 



^(-^-) - 2M.)ETTfg;^ + A^(")E(T^ 



{uW 



for some |^i(w)| < |A„(w)||Ai(u)|. Note that the first term on the right hand side 
equals 0. Hence, 



sup £(u, 9,g) > 



s^PueUo ^' 



-'EtiAUu) 



ueUo 
It now follows that 



2sup„gy^,(|ni/2A„(u)|-2) + 2sup„^, \n-y^A{i 



P[c < Sn < oo) > P( sup £{u, 6,g) > c\Sn < oo)P{Sn < oo) 
ueUo 

> P(sup \T{u)\ - Rni > 2c sup (|ni/2A„(u)|-2) + 2ci?„2, 

-Rril < £l-,Rn2 <£2\Sn< Oo)P(5„ < Oo), 



(A.3) 
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for some ei, £2 > to be chosen later, where i?„i = sup„ |n ^ X^ILi ^?('")~^(^)l 
and Rn2 = sup„ ^ \n^^/^Ai{u)\'^. Next, (A. 3) is bounded below by 

-1/2 



'(inf„gt/o|n''''A„H|>(2c)V2 



Rnl < £1, Rn2 < £2 



suPuGC/I^C")! -ei -2ce2 
5„ < oo)P(S„ < 00) 



-1/2 



S„ < 00 P(5„ < 00) 



>P(^mf„gyJni/2A„(„)| >(2c)i/2 

+P{fl„i < ei) + P(i?„2 < £2) - 2 
= P(5„ < 00) + 0(1), 

for £1, £2 > small enough, using (C2), (C3') and (A. 2). This finishes the proof. 

Proof of Theorem 3.1. 

For condition (CO), note that 

P{EL{x, e) = for some x £ Rx) 

= P(for some x e Rx,Yi - ^{Xi,6) > for all Xj G Jx 

or Y, - -/{Xi, e) <0 for all X, e J^) 
< P{e, > -S for all Xi < a or for aU Xi > a) 

+P(£j < S for all Xi <a or for all X^ > a) 

+P(sup \jixJ)^jix,eo)\>S) 

x£Rx 

-Pi +P2 +Pi +P2 +0(1) 
= 0(1), 

where S > 0, pi = P{e > -S), p2 = P(£ < S), l,, = Er=i -^(^^ ^ «) and 
kn = n — In- The latter equality follows from the fact that pi < I and P2 < 1 
for (5 > small enough, and the fact that kn,ln —* cxd since a belongs to the 
interior of Rx- Condition (CI) follows from Corollary 1.3 in [IG] (it is easily 
seen how I{Xi < x) can be replaced by I{Xi e J^) in the proof of that corollary 
(i = 1, . . . ,n)). For (C2); consider 



-i^/(X, eJ,)[K,-7(X„^)]2 

n 

n-^ J2 I{X, e Jx)[Y, - 7(X„ 0o)]' 



'ri 

-i^/(X, e J,)( 



(h{x^ 
de 



-2n-i Y, ^(^« e J,)[K, - 7(X„ 0o)] 



{0 - 0o)' 
97(^^,^i), 



4=1 



86 



-i^/(X, G J,)[r,-7(X„0o)]' + op(l), 



(A.4) 
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uniformly in a; G Rx, provided conditions (P1)-(P3) hold, and where 0i is 
between 6^0 and 6. To show that (A. 4) converges to T{x) uniformly in x, it 
suffices to show that the class of functions 

T = {(t, y) ^ I(t e J,..)[y - j{t, 0o)]' : x e Rx} 

is Glivenko-Cantelli. This is obvious, since the class of functions t -^ I{t € J^) 
is contained in the class of monotone and bounded functions into [0, 1], which is 
Glivenko-Cantelh (see [22], p. 149), and since the function (f, y) -^ [y — l[t^ ^o)]^ 
is independent of x. The fact that mix^Rx T{x) > and that sup^g^ T(x) < c» 
follows from condition (PI). We finally prove (C3). Note that the left hand side 
of the statement in condition (C3) is bounded by 

max \Y,-j{X,,9o)\+ max \jiX,,9) ~ jiX,,9o)\. 

l<i<n l<z<n 



The first expression above is op{n^^'^) (see Lemma 11.2 in [14]), whereas the 

d9 



second term is op{l) by the uniform boundedncss of "^L^' ' over x and 9 and 



since 9 — 9o = op(l). 

Proof of Theorem 3.2. 

We only need to show the validity of condition (C4). Write 



RP*{x)-Rt{x) = -n-V2^/(X, G Jx)[j{X,,9)^j{X,,9„)]V, 

-G*(x, 9„)n-^/^ ^[/i(X,;, K„ 9) - h{X,, Y,,9o)]V, 

-[G*(a;,0VG*(:r,0o)]n"'/'^/^(^,:,>;:,^o)14+op(l). 

The second term above is op(l) by condition (P4). The convergence to zero of 
the first term follows from the fact that by assumption (P5), 



n-V2 



^/(X, e Jx)[j{X,,e)~j{X,,9o)]V^ 

'"^ " d 

= {9- eofn-'/^ Y. ^(^' e J:c)-g^7{X^,9o)V, + op(l) = op(l), 

i—l 

since E[I{Xi G Jx)-§gl{^i,(^o)yi\ = 0- Finally, for the third term above note 
that n~^/^Y:tlHX^,Y,,9o)Vi 

= Op(l) and that G{xJ) - G{x,9o) = G{x,9o) - G{x,9q) + op{l). In a similar 
way as in the proof of Theorem 3.1 one can show that the class 

i^t ^ I{t e Jx)-g^jit,9o) : X e Rx} 
is Glivenko-Cantelli. Hence, the third term is also op(l). 
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Proof of Theorem 3.3. 

Condition (CO) can be shown in a similar way as in the proof of Theorem 3.1. 
Condition (CI) is an immediate consequence of Theorem 1 in [20]. For (C2), 
note that 

n-1 Y, iCP'X, < x)[Y, - 7(/3*X„ a)]2 

n 
i=\ 

+n-^x:^(/3o^-^^)[ '^^^i^""^T ("~"o)^ 

Z— 1 

-2n-if^/(/3*X, < x-)[r, -7(/3*X„ao)]^^^^^^^^(a-ao) 

n 

+n-i ^[/(/3*X, <x)- liPlX, < x)][Y, ~ 7(/3*X„d)]2 
1=1 

n 

+n-i^/(/3*X, < x){[Y,-j(pX,,a)]' ~ [F, - 7(/?S^z, «)]'}, 

i=l 

where ai is between ao and a. The first three terms above can be dealt with in 
a similar way as in the proof of Theorem 3.1. For the fourth term above, note 
that, since /3 — /3o = Op{n^^^^) (by condition (GLM2)), the number of z-valucs 
for which /(/3*X, <x)- I{[3lX, < x) 7^ is of the order 0((nloglogn)i/2) a.s. 
(by the law of iterated logarithm, see e.g. [15], p. 35). Hence, the fourth term is 
0((n-Moglogri.)i/2) a.s. The fifth term is Op{n-'^/'^), by conditions (GLMl), 
(GLM2) and (GLM3)(ii). Finally, to check (C3); a similar derivation as in the 
proof of Theorem 3.1 can be followed. 

Proof of Theorem 3.4. 

The proof can be established in a somewhat analogous way as for the paramet- 
ric model. We therefore restrict attention to the main points of difference. In a 
similar way as in the proof of Theorem 1 in [20], it can be easily shown that 
when one replaces the indicator I{(3*Xi G J^) by I{f3}^Xi S J^) in the formula of 
j^GLM*(^^^ (i.e. in the formulas of Q{Xi,Yi,u,9) and G{u,9)), the asymptotic 
distribution does not change. This so-obtained process can now be analyzed in 
a very similar way as the process R^*{x) in the proof of Theorem 3.2, which 
then shows the equivalence of R^^^*{u) and R^^^*{u), uniformly in u. 

Proof of Theorem 3.5. 

For condition (CO), a derivation similar to the one in the proof of Theorem 3.1 
can be given, using the fact that sup^ \9{w) — g{w)\ = op(l). Condition (CI) 



n 
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follows from Theorem 1 in [o]. For condition (C2), write 

n 

n 

n 
i=l 

The first term above can be treated as in the proof of Theorem 3.1. The sec- 
ond one is op(l) by using assumption (SV4) and the fact that sup^„ \fw{w) — 
fwiw)\ = op{l), while the negligeability of the third term follows again from 
condition {SV4) and from the uniform consistency of g{w) for all w G Rw- 
Finally, for (C3), note that 

supmax|/(X, e Jcc)fw{W,){Y,-g{W,))\ 

< max \jw{WC){Y^ ~ g(W,))| + max |[/m/(W,) - fwiW,)]iY, - g{W,))\ 

i i 

+ m^x\fw{W,mW,) - g{W,)]\ 

i 
= Op(l), 

by using again the same development as in the proof of Theorem 3.1 for the first 
term, and the uniform consistency of g and fw for the second and third term. 
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